Papers with attention alignment
Attention Alignment and Flexible Positional Embeddings Improve Transformer Length Extrapolation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for length extrapolation are tailored for natural language modeling, a task known to have strong recency bias. |
| Approach: | They propose two attention alignment strategies to improve T5's long-context utilization capability without fine-tuning. |
| Outcome: | The proposed methods improve the long-context utilization capability of T5 on language modeling, retrieval, multi-document question answering, and code completion tasks without any fine-tuning. |
ReLoop: “Seeing Twice and Thinking Backwards” via Closed-loop Training to Mitigate Hallucinations in Multimodal understanding (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for hallucination mitigation rely on external verification or post-hoc correction, lacking internal mechanism to validate outputs directly during training. |
| Approach: | They propose a unified closed-loop training framework that encourages multimodal consistency for cross-modal understanding in MLLMs. |
| Outcome: | The proposed framework encourages multimodal consistency for cross-modal understanding in MLLMs. |
Understanding Attention for Vision-and-Language Tasks (2022.coling-1)
Copied to clipboard
| Challenge: | Attention mechanism has been used in Vision-and-Language (VL) tasks to bridge the semantic gap between visual and textual clues. |
| Approach: | They conduct a comprehensive analysis on understanding the role of attention alignment by looking into attention score calculation methods and checking how it represents the visual region’s and textual token’s significance for the global assessment. |
| Outcome: | The attention score calculation methods represent visual region’s and textual token’s significance for the global assessment. |
ChartGaze: Enhancing Chart Understanding in LVLMs with Eye-Tracking Guided Attention Refinement (2025.emnlp-main)
Copied to clipboard
Ali Salamatian, Amirhossein Abaskohi, Wan-Cyuan Fan, Mir Rayat Imtiaz Hossain, Leonid Sigal, Giuseppe Carenini
| Challenge: | Chart question answering (CQA) is a key research challenge for large vision-language models . recent efforts focus on leveraging LVLMs directly on chart images . |
| Approach: | They propose a gaze-guided attention refinement that aligns image-text attention with human fixations to improve chart reasoning quality and interpretability. |
| Outcome: | The proposed approach improves answer accuracy and attention alignment yielding gains of up to 2.56 percentage points across multiple models. |
Self-Explaining Hate Speech Detection with Moral Rationales (2026.findings-acl)
Copied to clipboard
Francielle Vargas, Jackson Trager, Diego Alves, Matteo Guida, Surendrabikram Thapa, Berk Atıl, Daryna Dementieva, Andrew J Smart, Ameeta Agrawal
| Challenge: | Existing models for hate speech detection are opaque and rely on surface-level cues. Existing approaches often encode biases originating from training data and annotation processes. |
| Approach: | They propose a framework that integrates moral rationale supervision into training . they propose SMRA for self-explaining hate speech detection . |
| Outcome: | The proposed framework improves performance across binary hate speech detection and multi-label moral sentiment classification. |